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Abstract 

We propose a measure of divergence of probability distributions for quanti- 
fying the dissimilarity of two chaotic attractors. This measure is defined in 
terms of a generalized entropy. We illustrate our procedure by considering the 
effect of additive noise in the well known Henon attractor. Comparison of two 
Henon attractors for slighly different parameter values, has shown that the 
divergence has complex scaling structure. Finally, we show how our approach 
allows to detect non-stationary events in a time series. 
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Through the appropriate embedding procedures, strange attractors can be numerically 
approximated by a large sets of points, either from experimental time series (TS) or from 
numerical simulation of chaotic systems. Advances in nonlinear analysis of TS have made 
possible to identify and classify chaotic dynamical systems, to determine if a signal is deter- 
ministic or not, and to establish correlations where the traditional linear analysis were not 
sensitive. However, there are many situations where we need not a complete characteriza- 
tion of an attractor, but rather a quantitative way of comparing attractors. For instance, 
recently several authors have proposed to use some measures of dissimilarity of attractors to 
analyze non- stationary signals |E]|| and for TS classification 0. In some situations it could 
be important to quantify the difference of two attractors from the same chaotic dynamical 
systems, corresponding to slightly different parameters. The computation of the hierarchy 
of generalized dimensions does not help, because even if all dimensions of two fractal sets 
are equal, this does not guarantee that the two fractal objects are identical. In order to 
give a quantitative answer to these issues, a number of dissimilarity measures have been 
proposed in the literature [0,f|||. The quantitative comparison of attractors can be relevant 
in many different problems such as: numerical taxonomy of TS; to establish a criterion for 
stationarity; to study the numerical convergence of chaotic solutions; to evaluate the ef- 
fect of nonlinear noise reduction of noisy chaotic attractors, among other applications. For 
the above mentioned purposes we need a reliable way of comparing attractors rather than 
their detailed characterization. In this paper, we propose a divergence measure based on a 
generalized entropy function for quantifying the similarity of attractors. 

From the information theory viewpoint, the amount of uncertainty of the probability 
distribution (PD), pi, is defined in a general way by Hf [p.] = f [Pi] 0- There is not a 
unique information measure Hf. The more commonly used information measure or entropy 
function was introduced by Shannon [7] where f (p) = pin p. Generalized entropy f q has 
been postulated by Renyi || and Havrda-Charvat |J. Renyi's generalized entropy has been 
used to define a hierarchy of generalized dimensions [|10|| . Tsallis introduced the Havrda- 
Charvat entropy function to elaborate an non-extensive thermodynamics [JTT ] . Associated 



with an entropy function /, we have a divergence measure Df(p:p) between two PD pi and 



pi. A general divergence measure form associated to the /-entropy was given by Csiszar (12 

Df(p:p)=E(pif\f]+Pif\ P A)> (1) 

where / is a convex function and one imposes the condition / (1) = 0, that guarantees 
Df (p : p) = 0. Renyi's generalized entropy does not satisfy convexity. Fortunately, Havrda- 
Charvat entropy function fulfills this property. For this reason we will work here with the 
Havrda-Charvat entropy function. From here on we shall refer to the generalized divergence 
measure associated with the Havrda-Charvat entropy function, simply as the g-divergence, 
and will be denoted by D q . If we replace / by the function corresponding to the Shannon 
entropy, we obtain the well known Kullback-Leibler distance [[13] . 
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The function / corresponding to the Havrda-Charvat entropy is given by f q (p) 
(q — 1) _1 (p q — p). Then, the associated g-distance is given by 

D q (p:p) = (q- I)" 1 (Y.plP]- q + Y.Pl.p\~ q ~ 2) • 



(3) 



It is easy to show that D q (p : p) — > D\ (p : p) when q — > 1. The g-divergence measure 
is positive definite, has been made symmetric, and fulfills D q (p : p) = 0. Also D q (p : p) 
considered as a function of pi and pi is convex. We remark that D q is semi-metric, since it 
may not satisfy the triangular inequality. 

After these definitions, let us consider the divergence between two finite time series 
embedded in 7Z d , X = (x , . . . , xjy) and Y = (y , . . . , yjv). There are two well known ways 
of estimating the quantities (Q) and (|3|) from X and Y . The most straightforward, but also 
more expensive, is to use a box counting approach: one defines a partition U £ of the state 
space, with characteristic size e. Thus the probability to find a point of X (Y) in the ith 
box is Pi (pi). By counting the number of points rii in the box i, the probabilities Pi can be 
estimated as pi = rii/N, where iV is the total number of points. Some authors have refined 
this procedure, by adapting the size of the boxes depending on the local density |14| . 



On the other hand, a more efficient method of estimating (0) and ([3]) is by correlation 
sums 



15|j . Instead of taking a fixed mesh, one can calculate the probability P (x^, e) to find 
a d- dimensional point within a sphere of radius e centered around the Xj, with j = 1, . . . , M 
randomly chosen from the trajectory X. P(xj,e) is estimated by counting the number rij 
of points falling in the sphere of radius e centered at Xj, 

N 

P(x„e) = iV- 1 ^e(£-|x,-x,|) j = l,...,M. 
i=i 

In our case the expressions (H) and (§) can be rewritten as 
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D 1 (p:p,e) = M- 1 ( £ In 
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Ef =1 e( g -| Xi - yj |) 
£i=i©(£- |y*-yil) 



, (4) 
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(5) 

Where 0(x) is the step function which has the value 1 if x > and is otherwise, |xj — y^-j 
is the distance between Xj and y^. The sum is taken only for those i's and j's that are 
separated in time by more than B sampling times to avoid artifactural correlations |TB 



thus N' = N — d — B . Notice that the quantities @) and (§) are defined for two finite 
discrete probability distributions pi and pi only if pi > 0, and pi > 0, and if there is a one- 
to-one correspondence between the elements i In order to satisfy these requirements, we 
perform the summation in the box counting approach (Eq. (fj) and (||)) only over the boxes 
i that contain points from both X and Y (i.e. Pi > and pi > 0). In the sphere counting 
scheme (Eq. (|4]) and ([5])), we include in the summation over j only spheres which contain 
points both in X and Y and for this reason M decreases with e. 

Now, we shall present some examples of numerical computations of the q-divergence D q 
between two time series. We estimate the D q by means of the two before mentioned methods: 
the box counting (BC), and the sphere counting (SC). In the case of BC algorithm, for 
simplicity, we restrict our analysis to dimension d = 2. As our first example, we deal with 



trajectories of 10 000 points of the Henon model x n+ \ = 1 — ax\ + &x n -i, with parameters 
a = 1.4 and b = 0.3; cf. Kantz Q. The set X corresponds to the clean attractor, while 
the set Y consists of the same set of points plus an additive Gaussian noise. In Fig. 1 we 
present the mean value of D% versus e computed with the BC method. We computed the 
mean value over 5 realizations of Y with signal-to-noise ratio 77 = 20 dB []17|| . Of course, 
the divergence depends of the length scale e. By choosing a relatively small e, D q will 
pick up local differences between X and Y. However, taking e too small leads to poor 
statistics. For large e, we lose the small scale structure of the attractors that they become 
indistinguishable. We can see that D\ reaches a maximum at a values of e that will be 
denoted by e . The dashed curve corresponds to S — (Di)/a(Di) where () denotes the 
mean value over 5 realizations and a denote the standard deviation. We can see that the 
maximum value of D x presents very good statistics. 

In Fig. 2 we display the q dependence of D q (eq) for different noise level (rj = 30 dB 
in solid line, rj = 20 dB in dashed line and rj = 10 dB in dotted line). As shown in Fig. 
2, the parameter q is like a gain control parameter. The divergence of the two attractors 
increases with q. This fact can be used to detect small divergence as we will discuss in the 
last example. In Fig. 3 we illustrate the behavior of the mean value of D 2 versus e computed 
with SC method. We computed the mean value over 5 realization of Y for each level of noise. 
We can see that D 2 presents a maximum at eq which depends on the characteristic length 
scale at which the attractors differ. The e dependence of D q characterizes the relationship 
between the sets X and Y . In the case analyzed here, this length scale is related to the 
level of noise added to X. In Fig. 4 we display D 2 (eq) versus 77. This figure shows that the 
D 2 (eq) scales exponentially with the level of noise added to the signal. 

Now, we compare two clean Henon attractors for slightly different parameter values. In 
this case Y corresponds to parameters a = 1.35 and b = 0.31, while X remains the same. In 
Fig. 5 we display D\ computed with N = 50 000 points, embedding dimension d = 2, using 
the BC method (dotted line), using the SC method (dashed line), and with d = 3 using the 
BC method (solid line). When we compare these attractors, we find that the divergence 
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measure as a function of e exhibits a more complex scaling structure than showed in the 
previous example. This complex relationship between the attractors has not been reported 
in early studies PJIB|. 

Let us finally show how the g-distance D q can be used to detect non stationarity events 
in a TS. As numerical example, let us consider a generalized baker map defined by 

v n <a: u n+ i = (3u n , v n+1 = v n /a, 

v n - a 



v n > a : u n+1 = 0.5 + f3u n , v n+1 

1 — a 

For this map, the parameter (3 can be varied without changing the positive Lyapunov expo- 
nent. We generate a non stationary TS of length 8192 points with a = 0.4 and two values 
of (3. In the first 4096 iterations we use (3 = 0.6, and in the second 4096 iterations we set 
(3 = 0.8. We record u + v, then we subtract the mean value and normalize to unit variance 
separately each one of the two parts; cf. Schreiber [FJ]. Thus, we have a non stationary 
event in the middle of the TS that is very hard to detect, because observables like mean, 
variance and maximal Lyapunov exponent are constant by construction. Figure 6 shows 
D 2 (Si : Sj) and D 6 (Si : Sj), between two nearest non-overlapping segments Si and Sj of 



1000 points [r9|. This example shows also that the parameter q plays a role of a nonlinear 
gain parameter. We can see that for q = 6 the divergence was able to detect precisely when 
the small change in (3 ocurr, while for q = 2 we have a poor discrimination power. We used 
resolution e = 0.1155 in the computation. We have a window resolution with similar results 
for e in [0.075,0.15]. 

We have introduced the g-divergence as a measure of dissimilarity of two finite sets. 
Our approach is particularly useful for comparing attractors. We found that the divergence 
decreases exponentially when rj increases. Comparison of two Henon attractors for slighly 
different parameter values has shown that the g-divergence has a complex scaling structure. 
Also this tool promises to be useful for detecting non-stationary event in a TS, even in very 
hard conditions. Thus It will be seen that some interesting physical insight is gained by 
recourse to this type of dissimilarity measure. 
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FIG. 1. Solid Line: D\ (e) between the TS X generated by the Henon system and the TS 
Y generated by the same systems contaminated with noise (77 = 20 dB), computed with the BC 
scheme using N = 10 000 points and d = 2 (left axis). Dashed line: the statistic S (e) (right axis). 
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FIG. 2. The (/-distance D q (eo) between the TS X generated by the Henon system and the TS 
Y generated by the same systems contaminated with several level of noise as a function of the 
parameter q (solid line: SNR= 30 dB, dashed line: 7/ = 20 dB and dotted line 77 = 10 dB) . The 
calculation were performed with the BC scheme using N = 10 000 points and d = 2. 
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FIG. 3. Distance D2 between Henon system and itself contaminated with several level of noise 
(between 77 = 30 dB to rj = 7 dB). The calculation were performed with the SC scheme using 
N = 10 000 points, B = 20 and d = 2. 
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FIG. 4. Divergence D2 (eo) between Henon system and itself contaminated with noise as a 
function of the level of noise rj. 
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FIG. 5. Estimated values of D\ as a function of e for two time series of length N = 50 000, and 
B = 20 generated by slightly different Henon systems. Dotted line: using BC method with d = 2. 
Dashed line: using SC method with d = 2. Solid line: using SC method with d = 3. 




FIG. 6. Dq (solid line) and D2 (dashed line) between two non-overlapping subsequent segments 
of TS with a non-stationary event at 4096. 
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